Skip to content

[ddev] Retry agent check on transient failure to fix SNMP E2E flake#23646

Merged
Kyle-Neale merged 2 commits into
masterfrom
kyle.neale/ddev-agent-readiness-gate
May 12, 2026
Merged

[ddev] Retry agent check on transient failure to fix SNMP E2E flake#23646
Kyle-Neale merged 2 commits into
masterfrom
kyle.neale/ddev-agent-readiness-gate

Conversation

@Kyle-Neale
Copy link
Copy Markdown
Contributor

@Kyle-Neale Kyle-Neale commented May 8, 2026 •
edited
Loading

What does this PR do?

Wraps agent check invocations in ddev env agent with a bounded retry (3 attempts, 0.5s backoff).

Motivation

Mitigates the SNMP master.yml E2E flake (no valid check found) — a brief race in the existing E2E config-swap path where autodiscovery can deregister the check between the swap and the immediate agent check call. ~44% of recent master SNMP runs hit this; >99% of tests still pass within failing runs.

Review checklist (to be filled by reviewers)

  • Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
  • Add the qa/skip-qa label if the PR doesn't need to be tested during QA.
  • If you need to backport this PR to another branch, you can add the backport/<branch-name> label to the PR and it
    will automatically open a backport PR once this one is merged

@dd-octo-sts dd-octo-sts Bot added the ddev label May 8, 2026
@codecov
Copy link
Copy Markdown

codecov Bot commented May 8, 2026 •
edited
Loading

Codecov Report

❌ Patch coverage is 52.94118% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.85%. Comparing base (996b3d5) to head (7dbeb04).
⚠️ Report is 21 commits behind head on master.

Additional details and impacted files
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@datadog-datadog-prod-us1
Copy link
Copy Markdown
Contributor

datadog-datadog-prod-us1 Bot commented May 8, 2026 •
edited by datadog-prod-us1-4 Bot
Loading

Tests

✨ Fix all issues with BitsAI or with Cursor

⚠️ Warnings

🧪 1 Test failed

❄️ Known flaky: test_e2e_snmp_listener from test_e2e_snmp_listener.py   View in Datadog   (Fix with Cursor) 
Needed at least 1 candidates for 'datadog.snmp.check_duration', got 0
Expected:
        MetricStub(name='datadog.snmp.check_duration', type=0, value=None, tags=['autodiscovery_subnet:172.18.0.0/28', 'device_vendor:apc', 'firmware_version:2.0.3-test', 'loader:python', 'model:APC Smart-UPS 600', 'serial_num:test_serial', 'snmp_device:172.18.0.1', 'snmp_profile:apc_ups', 'ups_name:testIdentName'], hostname=None, device=None, flush_first_value=None)
Difference to closest:
        Expected tag snmp_device:172.18.0.1
        Found snmp_device:172.18.0.2

Similar submitted:
Score   Most similar
1.00    MetricStub(name='datadog.snmp.check_duration', type=0, value=0.18897485733032227, tags=['autodiscovery_subnet:172.18.0.0/28', 'device_vendor:apc', 'firmware_version:2.0.3-test', 'loader:python', 'model:APC Smart-UPS 600', 'serial_num:test_serial', 'snmp_device:172.18.0.2', 'snmp_profile:apc_ups', 'ups_name:testIdentName'], hostname='runnervmrc6n4', device=None, flush_first_value=False)
...

ℹ️ Info

No other issues found (see more)

❄️ No new flaky tests detected

🎯 Code Coverage (details)
• Patch Coverage: 52.94%
• Overall Coverage: 87.36%

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 7dbeb04 | Docs | Datadog PR Page | Give us feedback!

@Kyle-Neale Kyle-Neale force-pushed the kyle.neale/ddev-agent-readiness-gate branch 2 times, most recently from 0102e1e to ee648c2 Compare May 11, 2026 13:58
@Kyle-Neale Kyle-Neale changed the title [ddev] Wait for Agent cmd-server before returning from start() [ddev] Retry agent check on transient failure to fix SNMP E2E flake May 11, 2026
@Kyle-Neale Kyle-Neale force-pushed the kyle.neale/ddev-agent-readiness-gate branch 3 times, most recently from c3d50af to f277b75 Compare May 11, 2026 19:03
@Kyle-Neale Kyle-Neale force-pushed the kyle.neale/ddev-agent-readiness-gate branch from f277b75 to bd4527c Compare May 11, 2026 20:02
@Kyle-Neale Kyle-Neale marked this pull request as ready for review May 12, 2026 13:34
@Kyle-Neale Kyle-Neale requested a review from a team as a code owner May 12, 2026 13:34
chatgpt-codex-connector[bot]
chatgpt-codex-connector Bot reviewed May 12, 2026
View reviewed changes
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bd4527ce4b

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread ddev/changelog.d/23646.fixed Outdated
@Kyle-Neale @claude
fix(ddev): add trailing period to changelog entry
7dbeb04 
Per AGENTS.md, changelog entries must end with a period.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@dd-octo-sts
Copy link
Copy Markdown
Contributor

dd-octo-sts Bot commented May 12, 2026

⚠️ Recommendation: Add qa/skip-qa label

This PR does not modify any files shipped with the agent.

To help streamline the release process, please consider adding the qa/skip-qa label if these changes do not require QA testing.

@dd-octo-sts
Copy link
Copy Markdown
Contributor

dd-octo-sts Bot commented May 12, 2026

Validation Report

All 20 validations passed.

Show details
Validation Description Status
agent-reqs Verify check versions match the Agent requirements file ✅
ci Validate CI configuration and Codecov settings ✅
codeowners Validate every integration has a CODEOWNERS entry ✅
config Validate default configuration files against spec.yaml ✅
dep Verify dependency pins are consistent and Agent-compatible ✅
http Validate integrations use the HTTP wrapper correctly ✅
imports Validate check imports do not use deprecated modules ✅
integration-style Validate check code style conventions ✅
jmx-metrics Validate JMX metrics definition files and config ✅
labeler Validate PR labeler config matches integration directories ✅
legacy-signature Validate no integration uses the legacy Agent check signature ✅
license-headers Validate Python files have proper license headers ✅
licenses Validate third-party license attribution list ✅
metadata Validate metadata.csv metric definitions ✅
models Validate configuration data models match spec.yaml ✅
openmetrics Validate OpenMetrics integrations disable the metric limit ✅
package Validate Python package metadata and naming ✅
readmes Validate README files have required sections ✅
saved-views Validate saved view JSON file structure and fields ✅
version Validate version consistency between package and changelog ✅

View full run

@Kyle-Neale Kyle-Neale added this pull request to the merge queue May 12, 2026
Merged via the queue into master with commit 89441e6 May 12, 2026
362 of 363 checks passed
@Kyle-Neale Kyle-Neale deleted the kyle.neale/ddev-agent-readiness-gate branch May 12, 2026 21:14
@dd-octo-sts dd-octo-sts Bot added this to the 7.79.0 milestone May 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@chatgpt-codex-connector chatgpt-codex-connector[bot] chatgpt-codex-connector[bot] left review comments

@steveny91 steveny91 steveny91 approved these changes

Assignees

No one assigned

Labels

ddev team/agent-integrations

Projects

None yet

Milestone

7.79.0

Development

Successfully merging this pull request may close these issues.

2 participants

@Kyle-Neale @steveny91